Experimental investigation on the complexity-performance relations in Multilayer Perceptrons

This paper describes experimental investigations for exploring the dependence of Neural Networks behavior and capabilities on their complexity. Characteristic behavior patterns are worked out through an artificial problem. We analyze in particular the dependency of overfitting on NN complexity, characterize the bias-variance evolution of the error, and the effects of two regularization techniques. Results put in evidence the discrepancy between effective and theoretical capabilities of MLPs. It is also stressed that rules derived from general statistical results must be used with care in the case of neural networks.